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COMMENTS  ON  KHACHIAN’S  ALGORITHM  FOR  LINEAR  PROGRAMMING 

by 

George  B.  Dantsig* 


Jack  Edmonds  once  defined  a  “good"  algorithm  as  one  that  can  solve  any 
problem  within  a  class  in  a  polynomial  bounded  time  where  each  factor  of  each 
term  of  the  polynomial  is  some  parameter  either  expressing  the  size  of  the  problem 
or  the  number  of  digits  needed  to  store  the  data  of  the  problem.  The  point  of  this 
little  essay,  is  this:  Solving  in  a  polynomially  bounded  time  says  little. 

A  POLYNOMIAL  BOUND  CAN  BE  A  BIG  NUMBER 


The  expected  number  of  steps  to  find  a  feasible  solution  to  a  linear  program 
using  Phase  1  of  the  Simplex  Method,  for  moderately  sized  problems, is  conjectured 
to  be,  of  the  order 

o  •  m  STEPS, 


where  m  is  the  number  of  equations  and  a  is  typically  2  to  3  (or  4  to  8  for  an 
optimal  solution  using  both  Phase  I  and  II).  Thousands  of  linear  programs  are 
solved  each  day  using  some  variant  of  the  simplex  method — a  value  of  a  >  4  is 
rarely  seen.  The  effort  to  do  each  pivot  step  is  of  order  m2  but,  because  most 
coefficients  of  the  matrix  are  usually  zero,  the  work  to  do  a  pivot  cau  be  reduced 
to  a  fraction  of  m2. 

Even  for  problems  involving  1000  equations  and  3000  variables  ,  o  appears 
to  be  small.  It  w  conjectured  for  n  large  relative  to  m,  that  the  expected  value  of 
a  will  grow  slowly,  say 

exp(o)  <  loga(2  +  £)  . 
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Peter  GAcs  and  Laszlo  LovAsz  state  in  [2]  that  the  effort  involved  to  solve  a 
problem  by  Khacbian’s  algorithm,  is  of  the  order  6L(n-{- 1)2  steps.  Richard  Stone 
has  pointed  out  that  the  correct  value  is  4L(fl-|- 1)2.  For  convenience  below  I  have 
rounded  this  down  to  4LrS3  steps.  The  work  of  each  step  (which  is  not  a  pivot 
operation)  is  of  order  fl2;  it  can  be  thought  of  as  comparable  to  a  simplex  iteration. 
Moreover,  the  devices  that  have  been  so  successful  in  reducing  the  work  of  each 
iteration  of  the  Simplex  Method  appear  applicable  to  reducing  the  work  of  each 
step  of  the  Khachian  method.  If  coefficients  range  in  value  by  10s  and  are  given 
to  5  significant  decimal  figures,  then  L  is  about  30m  n.  This  giveB  as  an  upper 
bound  for  the  number  of  steps  for  the  Khachian  algorithm: 

120  m  n3  STEPS. 

As  polynomials  go,  this  is  of  surprisingly  low  order — a  truly  remarkable  theoretical 
result  and  the  beginning  of  a  promising  and  exciting  new  area  of  research.  It  shows 
the  essentially  algebraic  character  of  linear  inequality  systems  in  spite  of  the  fact 
that  the  number  of  extreme  points  can  be  exponential  in  m  and  n. 

Whether  or  not  the  algorithm  is  practical  depends  on  the  size  of  this  bound  for 
moderately  large  linear  programming  problems  of  interest.  The  example  discussed 
below  is  typical. 

At  Stanford  we  have  developed  a  dynamic  linear  programming  model,  called 
PILOT,  for  Energy /Economic  planning.  It  is  used  by  the  Electric  Power  Research 
Institute  in  their  planning  for  the  Utility  Industry  and  by  the  Department  of 
Energy  for  their  long  range  studies.  Case  studies  (scenarios)  usually  require  5  to 
10  minutes  of  CPU  time  on  the  IBM  370-168  if  started  with  a  good  guess.  From  a 
"cold  start”  it  can  take  up  to  1  hour  of  CPU  time.  Because  the  problem  is  dynamic 
in  structure,  the  number  of  pivot  steps  seems  to  run  a  little  higher  on  the  average 
—  about  3.5  m  for  each  phase  of  the  Simplex  Method. 

For  the  PILOT  Model: 

m  =  1000  ,  n  —  3000 
or 

n  —  2000  ,  m  ==  3000 
The  comparative  number  of  steps  are: 


KHACHIAN:  120mn3  STEPS 
SIMPLEX:  3.5m  STEPS  — 


—  120  X  3000  X  20003 
3.5  X  103  STEPS  . 


2.88  X1015  STEPS 
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For  a  1000  equation  system  of  the  sparsity  of  PILOT,  each  step  takes  about 
1/2  second  on  the  IBM  370-188  or  at  the  rate  of  6.3  X  107  steps  per  year.  Thus 
the  estimated  and  actual  times  to  solve  this  problem  are 

KHACfflAN:  2.88  X  1015  STEPS  «=  50, 000, 000  years 

SIMPLEX:  3.5  X  103  STEPS  =s=  30  minutes  . 

This  comparison  is,  of  course,  unfair.  The  upper  bound  given  by  Khachian 
is  not  tight  and  a  tight  bound  for  the  worst  case  could  turn  out  to  be  a  much 
smaller  number.  Walter  Murray  has  looked  into  the  way  that  the  ellipsoids  adjust 
and  contract  in  volume.  Each  ellipsoid  appears  to  be  a  slight  perturbation  of  its 
predecessor.  Because  of  this,  Murray  believes  that  the  expected  number  of  Bteps 
(for  problems  of  the  same  dimensions  and  size  of  the  coefficients)  will  not  be  much 
different  from  that  of  the  worst  case  [4]. 

There  is  a  great  deal  of  interest  in  the  Khachian  algorithm.  One  can  expect 
a  reexamination  of  similar  previously  proposed  algorithms  that  converge  in  the 
limit  to  a  feasible  solution.  Restated  in  terms  of  integer  coefficients,  a  finiteness 
proof  with  a  polynomial  bound  might  also  be  obtained  perhaps  some  will  turn  out 
to  have  practical  upper  bound  and  expected  value  estimates.  For  a  discussion  of 
this  possibility  see  [4]. 

It  has  been  suggested  by  N.  Zadeh,  Alan  Hoffman  and  othera,  that  the  Simplex 
Algorithm  be  reexamined  to  see  if  it  too  has  polynomial  bound  under  the  assump¬ 
tions  that  the  class  of  linear  programs  considered  has: 

(1)  integer  coefficients,  and 

(2)  the  maximum  of  the  absolute  value  of  coefficients  is  less  than  a  constant 
independent  of  m  and  n. 

Examples  have  been  constructed  by  V.  Klee  and  G.  Minty  [3],  P.  Wolfe,  and  N. 
Zadeh  such  that  the  number  of  Bteps  grows  exponentially  but  these  do  not  satisfy 
(1)  and  (2)  above. 

In  summary,  the  existence  of  an  algorithm  with  a  polynomial  bound  of  120mn3 
steps,  each  step  requiring  about  n3  operations,  is  an  important  theoretical  result. 
Unfortunately  a  polynomial  bound  does  not  imply  a  “good  algorithm” .  To  qualify 
as  good,  the  bound  must  not  be  too  high  for  practical  problems  of  interest  such  as 
those  routinely  solved  by  the  Simplex  Method.  The  effect  of  the  Khachian  result 
will  be  to  intensify  the  research  to  find  an  algorithm  with  a  more  practical  bound 
like  2(rH — n)  steps,  each  step  requiring  about  (m — n)3  operations  (the  empirically 
observed  rough  average  for  finding  a  feasible  solution  using  the  Simplex  Method). 
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Khachian's  polynomial  bound  for  finding  a  feasible  solution  to  a  relaxed 
linear  program  is  an  important  theoretical  result.  Unfortunately,  a 
polynomial  bourn)  does  not  imply  a  good  algorithm  because  such  a  bound  could 
b*  too  large  for  problems  of  practical  interest.  For  example,  using  the 
formulae  in  the  original  paper,  practical  problems  with  3000  non-negative 
variables  and  1000  equations  (which  are  solved  under  one-half  hour  on  IRM 
170-168  using  the  simplex  method)  would  involve  over  40^:  iterations  and  would 
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